--- title: Sensors keywords: fastai sidebar: home_sidebar summary: "Reads and packages IMU/GPS/Pressure-Humidity-Temperature sensor data over UART. Used for georectification. " description: "Reads and packages IMU/GPS/Pressure-Humidity-Temperature sensor data over UART. Used for georectification. " nb_path: "nbs/07_sensors.ipynb" ---
{% include tip.html content='This module can be imported using from openhsi.sensors import *' %}{% include warning.html content='Still experimental. Stay tuned. ' %}
The OpenHSI camera requires motion to generate 2D spatial datacubes. Yet, motion also introduces other artefacts that need georectification. To correct for spurious motion, we need to collect absolute orientation and geolocation of the camera simultaneously with the camera capture. This is where this module comes in. An IMU, GPS, and Pressure/Humidity/Temperature sensor needs to be read and recorded.
A Teensy 4.0 operates all the sensors, and devices using a Real Time Operating System (RTOS) like cooperative scheduler to run each component update at the desired frequency. By reducing the I/O, and CPU load on the main development board (the Raspberry Pi 4 with 8 GB RAM), the sensor updates are offloaded to a microcontroller with a real time clock to sync and timestamp each sensor measurement. The whole thing is assembled onto a PCB that stacks with the Raspberry Pi 4 and battery hat.
| Component | Rate (Hz) | Info |
|---|---|---|
| Teensy 4.0 (uC) | 24 MHz Clock | --- |
| NEO 9N (GPS) | 20 Hz | I2C @400 kHz (takes ~20 ms per update) |
| BNO055 (IMU) | 100 Hz | I2C_1 @400 kHz |
| BME280 (Air) | 100 Hz | I2C_1 @400 kHz |
| DS3231 (RTC) | 100 Hz | I2C_1 @400 kHz |
| XBee | 1 Hz | UART_1 @115,200 Hz (~2 ms per update) |
| Raspberry Pi 4 | packets @100 Hz | UART @921,600 Hz (~0.8 ms per update) |
| Start button | 4 Hz poll | button linked to LED notifying status |
An XBee is also programmed to check sensor status remotely during operation. This could be useful to diagnose any issues without being physically connected to the microcontroller. A basic streaming dashboard is included.
Each data packet contains timestamped sensor data. The item fields are then extracted from the raw binary serial stream. {% include note.html content='The Teensy runs a 32 bit Cortex-M7 so serial packets are padded. In other words, the data struct is padded in contiguous memory so a byte variable followed by float variable will include 3 unused bytes in-between so things are packed as 32 bits at a time. ' %}
The data packet is sent as a C struct so we need to decode the binary stream and interpret each byte as the corresponding C type. In each packet, there are status bytes to indicate which sensor has been updated.
It's fairly safe to assume that all data saving will occur on a separate storage device. I tested this with an SSD mounted over USB.
{% include tip.html content='The serial port for Raspberry Pi is "/dev/serial0". For the Jetson, it is "/dev/ttyTHS0".' %}{% include note.html content='GPS PPS callbacks are experimental. I haven’t found a way to use them effectively.' %}
Let's now test this using simulated ancillary sensor data packets.
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '.')
ss.packets = []
for i in tqdm(range(77)):
ss.packets.append(collect_sim(rtc_offset_ms=150))
time.sleep(0.01)
#ss.save()
#hide_output
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '/media/pi/fastssd')
ss.master_loop()
#hide_output
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '/media/pi/fastssd',
cam_name="FlirCamera")
ss.master_loop(n_lines=256,
processing_lvl=2,
json_path="/media/pi/fastssd/cals/OpenHSI-FLIR01/OpenHSI-FLIR01_settings_Mono8_bin1.json",
pkl_path="/media/pi/fastssd/cals/OpenHSI-FLIR01/OpenHSI-FLIR01_calibration_Mono8_bin1.pkl",
preconfig_meta=None,
ssd_dir="/media/pi/fastssd",
switch_pin=17)
#hide_output
sd = SensorDashboard()
sd()
{% include warning.html content='Start/Pause button not working!' %}
#hide_output
sd.run()
The SensorDashboard will save the data coming in which can be accessed in a pd.DataFrame. Here is some experimental data with noise added to the latitude/longitude points so the ESRI map loads.
sd.data_df.head(10)